Dialogue Annotation for Language Systems Evaluation
نویسندگان
چکیده
The evaluation of Natural Language Processing (NLP) systems is still an open problem demanding further research progress from the research community to establish general evaluation frameworks. In this paper we present an experimental multilevel annotation process to be followed during the testing phase of Spoken Language Dialogue Systems (SLDSs). Based on this process we address some issues related to an annotation scheme of evaluation dialogue corpora and particular annotation tools and processes.
منابع مشابه
Segmented and Unsegmented Dialogue-Act Annotation with Statistical Dialogue Models
Dialogue systems are one of the most challenging applications of Natural Language Processing. In recent years, some statistical dialogue models have been proposed to cope with the dialogue problem. The evaluation of these models is usually performed by using them as annotation models. Many of the works on annotation use information such as the complete sequence of dialogue turns or the correct ...
متن کاملRecent Approaches to Arabic Dialogue Acts Classifications
Building Arabic dialogue systems (Spoken or Written) has gained an increasing interest in the last few. For this reasons, there are more interest for Arabic dialogue acts classification task because it a key player in Arabic language understanding to building this systems. This paper describes the results of the recent approaches of Arabic dialogue acts classifications and covers Arabic dialogu...
متن کاملA Method for Recognizing Temporal Expressions in Estonian Natural Language Dialogue Systems
Extraction of temporal expressions from an input text is an important step in natural language processing tasks. Automated extraction of temporal expressions can be used in dialogue systems where temporal constraints need to be enforced. The paper proposes an algorithm for processing temporal information in natural language. The algorithm was implemented as a standalone rule-based temporal expr...
متن کاملIdentifying Effective Moves in Tutoring: On the Refinement of Dialogue Act Annotation Schemes
The rich natural language dialogue that is exchanged between tutors and students has inspired many successful lines of research on tutorial dialogue systems. Yet, today’s tutorial dialogue systems do not regularly achieve the same level of student learning gain as has been observed with expert human tutors. Implementing models directly informed by, and even machine-learned from, human-human tut...
متن کاملTransfer of Corpus-Specific Dialogue Act Annotation to ISO Standard: Is it worth it?
Spoken conversation corpora often adapt existing Dialogue Act (DA) annotation specifications, such as DAMSL, DIT++, etc., to task specific needs, yielding incompatible annotations; thus, limiting corpora re-usability. Recently accepted ISO standard for DA annotation – Dialogue Act Markup Language (DiAML) – is designed as domain and application independent. Moreover, the clear separation of dial...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2000